Modern computing often use models, including machine learning models (e.g., neural network models), to produce a desired outcome (i.e., an analysis goal or analysis topic), given a dataset. Organizations frequently devote large amounts of resources to training these models and generate large numbers of models to analyze many large datasets. In many cases, the datasets comprise the same or similar data, and the models may perform similar functions or possess similar features (e.g., the same type of hyperparameter). Organizations may wish to use or generate a model to perform a new task (i.e., to perform a task it was not previously trained to perform). For example, the new task may be to analyze a new dataset or to produce a new outcome (analysis result).
As the numbers of models and datasets grow, it can become difficult to organize models and datasets in a meaningful way. As a result, computing systems may inefficiently spend time training new models when existing models are available. Further, computing systems may be unable to use information related to existing models to identify ways to improve model performance.
Model “brittleness” can cause problems when training a model to perform a new task. A “brittle” model is a model that may fail to converge during training. For example, a brittle model may work well for identifying faces in one person's photo album but may not work well for another person's photo album, without extensive retraining. In some cases, it can be difficult or impossible to train brittle models without human supervision (e.g., training models to generate synthetic data from sensitive data that human users cannot access). During training, brittle models may converge to a sub-optimal state and/or may converge slowly. For example, a model may converge to a model accuracy that is too low. In some cases, brittle models may fail to converge during training (e.g., the model may oscillate between two model states at each training step). Brittle models may need to be retrained to each newly received dataset. In many cases, it may not be apparent whether a model is brittle, without time consuming and costly training efforts.
Therefore, in view of the shortcomings and problems with conventional approaches to training models, there is a need for rapid, low-cost, unconventional systems that identify problems with model performance, including model brittleness, and improve model performance.